• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Chen Maotang, Zheng Sheng’an, You Litong, Wang Jingyu, Yan Tian, Tu Yaofeng, Han Yinjun, Huang Linpeng. A Distributed Persistent Memory File System Based on RDMA Multicast[J]. Journal of Computer Research and Development, 2021, 58(2): 384-396. DOI: 10.7544/issn1000-1239.2021.20200369
Citation: Chen Maotang, Zheng Sheng’an, You Litong, Wang Jingyu, Yan Tian, Tu Yaofeng, Han Yinjun, Huang Linpeng. A Distributed Persistent Memory File System Based on RDMA Multicast[J]. Journal of Computer Research and Development, 2021, 58(2): 384-396. DOI: 10.7544/issn1000-1239.2021.20200369

A Distributed Persistent Memory File System Based on RDMA Multicast

Funds: This work was supported by the National Key Research and Development Program of China (2018YFB1003302) and the SJTU-Huawei Innovation Research Lab Project (FA2018091021-202004).
More Information
  • Published Date: January 31, 2021
  • The development of persistent memory and remote direct memory access(RDMA) provides new opportunities for designing efficient distributed systems. However, the existing RDMA-based distributed systems are far from fully exploiting RDMA multicast capabilities, which makes them difficult to solve the problem of multi-copy file data transmission in one-to-many transmission, degrading system performance. In this paper, a distributed persistent memory and RDMA multicast transmission based file system(MTFS) is proposed. It efficiently transmits data to different data nodes by the low-latency multicast transmission mechanism, which makes full use of the RDMA multicast capability, hence avoiding high latency due to multi-copy file data transmission operations. To improve the flexibility of transmission operations, a multi-mode multicast remote procedure call(RPC) mechanism is proposed, which enables the adaptive recognition of RPC requests, and moves transmission operations out of the critical path to further improve transmission efficiency. MTFS also provides a lightweight consistency guarantee mechanism. By designing a crash recovery mechanism, a data verification module and a retransmission scheme, MTFS is able to quickly recover from a crash, and achieves file system reliability and data consistency by error detection and data correction. Experimental results show that MTFS has greatly increased the throughput by 10.2-219 times compared with GlusterFS. MTFS outperforms NOVA by 10.7% on the Redis workload, and achieves good scalability in multi-thread workloads.
  • Related Articles

    [1]Jiang Tianyang, Zhang Guangyan, Li Zhiyue. Survey on Concurrency Control Protocols of In-Memory Transactions[J]. Journal of Computer Research and Development, 2022, 59(4): 721-736. DOI: 10.7544/issn1000-1239.20210231
    [2]Chen Bo, Lu Youyou, Cai Tao, Chen Youmin, Tu Yaofeng, Shu Jiwu. A Consistency Mechanism for Distributed Persistent Memory File System[J]. Journal of Computer Research and Development, 2020, 57(3): 660-667. DOI: 10.7544/issn1000-1239.2020.20190074
    [3]Chen Youmin, Zhu Bohong, Han Yinjun, Tu Yaofeng, Shu Jiwu. A Hybrid Approach for Managing Data Pages in Persistent Memory File Systems[J]. Journal of Computer Research and Development, 2020, 57(2): 281-290. DOI: 10.7544/issn1000-1239.2020.20190574
    [4]Chen Youmin, Lu Youyou, Luo Shengmei, Shu Jiwu. Survey on RDMA-Based Distributed Storage Systems[J]. Journal of Computer Research and Development, 2019, 56(2): 227-239. DOI: 10.7544/issn1000-1239.2019.20170849
    [5]Zhou Jiang, Wang Weiping, Meng Dan, Ma Can, Gu Xiaoyan, Jiang Jie. Key Technology in Distributed File System Towards Big Data Analysis[J]. Journal of Computer Research and Development, 2014, 51(2): 382-394.
    [6]Li Zhen, Tian Junfeng, and Yang Xiaohui. Program Behavior Monitoring Based on System Call Attributes[J]. Journal of Computer Research and Development, 2012, 49(8): 1676-1684.
    [7]Zheng Liping, Chan Bin, Wang Wenping, Liu Xiaoping, Cao Li, Kuang Zhengzheng. Remote Visualization Based on Distributed Rendering Framework[J]. Journal of Computer Research and Development, 2012, 49(7): 1438-1449.
    [8]Chen Hao, Peng Cuifen, Sun Jianhua, and Shi Lin. XenRPC:Design and Implementation of Security VM Remote Procedure Call[J]. Journal of Computer Research and Development, 2012, 49(5): 996-1004.
    [9]Huang Hua, Zhang Jiangang, Xu Lu. Distributed Layered Resource Management Model in Blue Whale Distributed File System[J]. Journal of Computer Research and Development, 2005, 42(6): 1034-1038.
    [10]Yang Dezhi, Huang Hua, Zhang Jiangang, Xu Lu. BWFS: A Distributed File System with Large Capacity, High Throughput and High Scalability[J]. Journal of Computer Research and Development, 2005, 42(6): 1028-1033.
  • Cited by

    Periodical cited type(4)

    1. 孙辽东,王超,陈培,王德奎,李世刚,张书博,荆荣讯,王文潇. 基于云原生的人工智能训练业务监控系统设计. 信息技术与信息化. 2024(01): 188-192 .
    2. 梁晨,陈昊,杨舟,曲国远. 机载嵌入式超算平台RDMA技术研究及实现. 航空电子技术. 2023(04): 36-42 .
    3. 许贤慧,王淑营,曾文驱. 面向工程数据检索的ElasticSearch索引优化策略. 计算机与现代化. 2022(02): 79-84+119 .
    4. 彭拥军. 基于XMLRPC的分布式网络安全管理系统设计. 信息与电脑(理论版). 2021(20): 205-207 .

    Other cited types(9)

Catalog

    Article views (1361) PDF downloads (568) Cited by(13)

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return